17 research outputs found
Intrinsic Probing through Dimension Selection
Most modern NLP systems make use of pre-trained contextual representations
that attain astonishingly high performance on a variety of tasks. Such high
performance should not be possible unless some form of linguistic structure
inheres in these representations, and a wealth of research has sprung up on
probing for it. In this paper, we draw a distinction between intrinsic probing,
which examines how linguistic information is structured within a
representation, and the extrinsic probing popular in prior work, which only
argues for the presence of such information by showing that it can be
successfully extracted. To enable intrinsic probing, we propose a novel
framework based on a decomposable multivariate Gaussian probe that allows us to
determine whether the linguistic information in word embeddings is dispersed or
focal. We then probe fastText and BERT for various morphosyntactic attributes
across 36 languages. We find that most attributes are reliably encoded by only
a few neurons, with fastText concentrating its linguistic structure more than
BERT.Comment: To appear EMNLP 202
Generalizing Backpropagation for Gradient-Based Interpretability
Many popular feature-attribution methods for interpreting deep neural
networks rely on computing the gradients of a model's output with respect to
its inputs. While these methods can indicate which input features may be
important for the model's prediction, they reveal little about the inner
workings of the model itself. In this paper, we observe that the gradient
computation of a model is a special case of a more general formulation using
semirings. This observation allows us to generalize the backpropagation
algorithm to efficiently compute other interpretable statistics about the
gradient graph of a neural network, such as the highest-weighted path and
entropy. We implement this generalized algorithm, evaluate it on synthetic
datasets to better understand the statistics it computes, and apply it to study
BERT's behavior on the subject-verb number agreement task (SVA). With this
method, we (a) validate that the amount of gradient flow through a component of
a model reflects its importance to a prediction and (b) for SVA, identify which
pathways of the self-attention mechanism are most important.Comment: Long paper accepted at ACL 202
An Ordinal Latent Variable Model of Conflict Intensity
For the quantitative monitoring of international relations, political events
are extracted from the news and parsed into "who-did-what-to-whom" patterns.
This has resulted in large data collections which require aggregate statistics
for analysis. The Goldstein Scale is an expert-based measure that ranks
individual events on a one-dimensional scale from conflictual to cooperative.
However, the scale disregards fatality counts as well as perpetrator and victim
types involved in an event. This information is typically considered in
qualitative conflict assessment. To address this limitation, we propose a
probabilistic generative model over the full
subject-predicate-quantifier-object tuples associated with an event. We treat
conflict intensity as an interpretable, ordinal latent variable that correlates
conflictual event types with high fatality counts. Taking a Bayesian approach,
we learn a conflict intensity scale from data and find the optimal number of
intensity classes. We evaluate the model by imputing missing data. Our scale
proves to be more informative than the original Goldstein Scale in
autoregressive forecasting and when compared with global online attention
towards armed conflicts
A Measure-Theoretic Characterization of Tight Language Models
Language modeling, a central task in natural language processing, involves
estimating a probability distribution over strings. In most cases, the
estimated distribution sums to 1 over all finite strings. However, in some
pathological cases, probability mass can ``leak'' onto the set of infinite
sequences. In order to characterize the notion of leakage more precisely, this
paper offers a measure-theoretic treatment of language modeling. We prove that
many popular language model families are in fact tight, meaning that they will
not leak in this sense. We also generalize characterizations of tightness
proposed in previous works.Comment: 25 pages; ACL 2023 camera read
Towards Verifiable Text Generation with Symbolic References
Large language models (LLMs) have demonstrated an impressive ability to
synthesize plausible and fluent text. However they remain vulnerable to
hallucinations, and thus their outputs generally require manual human
verification for high-stakes applications, which can be time-consuming and
difficult. This paper proposes symbolically grounded generation (SymGen) as a
simple approach for enabling easier validation of an LLM's output. SymGen
prompts an LLM to interleave its regular output text with explicit symbolic
references to fields present in some conditioning data (e.g., a table in JSON
format). The references can be used to display the provenance of different
spans of text in the generation, reducing the effort required for manual
verification. Across data-to-text and question answering experiments, we find
that LLMs are able to directly output text that makes use of symbolic
references while maintaining fluency and accuracy.Comment: 46 pages, 4 figures, 6 table
UniMorph 4.0:Universal Morphology
The Universal Morphology (UniMorph) project is a collaborative effort providing broad-coverage instantiated normalized morphological inflection tables for hundreds of diverse world languages. The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema. This paper presents the expansions and improvements made on several fronts over the last couple of years (since McCarthy et al. (2020)). Collaborative efforts by numerous linguists have added 67 new languages, including 30 endangered languages. We have implemented several improvements to the extraction pipeline to tackle some issues, e.g. missing gender and macron information. We have also amended the schema to use a hierarchical structure that is needed for morphological phenomena like multiple-argument agreement and case stacking, while adding some missing morphological features to make the schema more inclusive. In light of the last UniMorph release, we also augmented the database with morpheme segmentation for 16 languages. Lastly, this new release makes a push towards inclusion of derivational morphology in UniMorph by enriching the data and annotation schema with instances representing derivational processes from MorphyNet
Intrinsic Probing through Dimension Selection
Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of tasks. Such high performance should not be possible unless some form of linguistic structure inheres in these representations, and a wealth of research has sprung up on probing for it. In this paper, we draw a distinction between intrinsic probing, which examines how linguistic information is structured within a representation, and the extrinsic probing popular in prior work, which only argues for the presence of such information by showing that it can be successfully extracted. To enable intrinsic probing, we propose a novel framework based on a decomposable multivariate Gaussian probe that allows us to determine whether the linguistic information in word embeddings is dispersed or focal. We then probe fastText and BERT for various morphosyntactic attributes across 36 languages. We find that most attributes are reliably encoded by only a few neurons, with fastText concentrating its linguistic structure more than BERT